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Abstract 

In this paper, we propose an original quality model for 
websites, developed starting from existing proposals 
and general requirements for quality models. Different 
approaches were defined to evaluate websites quality, 
but most of them were created for specific web 
domains (cultural, academic, tourist, enterprise 
environment, etc) or they consider only some web 
quality dimension becoming vertical models. Main goal 
of our work is to extrapolate a general and holistic 
model, easy to apply and scalable for different 
domains, aimed at helping web designers to develop 
accurate and quick websites evaluation, to facilitate the 
collaboration with the management and to support, 
thanks to different quality levels, their proposals to the 
customers. 
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Introduction 
Software Engineering gave and continues to give a 
great importance to the concept of quality and its 


related operations. In fact, in the last years, several 
processes and procedures were developed, until the 
definition of important standard characteristics for 
software product quality [3], including six main 
features (Functionality, Reliability, Usability, Efficiency, 
Maintainability and Portability) and four features about 
user’s point of view (Effectiveness, Productivity, Safety 
and Satisfaction). Moreover, ISO (International 
Standards Organization) distinguished these 
characteristics according to three views (developers’ 
view, Manager's view and users’ view) and so classified 
them in three levels of quality: internal, external and in 
use. Meanwhile, there was an important development 
of Web Engineering, that found its basis on the same 
concepts and principles but moved the focus from 
offline to online world. This increasing importance 
about the web applications needs more attention on 
user’s experience than on the process phases, because 
user’s perception and usability aspects can influence 
business possibility. Most of the living websites don’t 
respect neither the basic quality principles. This lack of 
attention is due to several reasons, such as the easy 
use of web-oriented languages, the rapid evolution of 
technologies, the tolerance of browsers to display also 
incorrect code, the presence of many web developers 
without specific background or knowledge and the 
frequent and bad custom to assign few time and money 
to this aim. Fortunately, evaluating websites quality has 
became a more and more discussed and analysed issue 
in the last ten years. 


Related works 

Several quality models or _ usability-focused 
approaches derived from computer-human interaction 
research were proposed. Some of them are based on 
ISO or IEEE (Institute of Electrical and Electronics 


Engineers) standard metrics, some are instead 
designed beginning by marketing researches or 
business interests and defined for specific or domain- 
dependent websites, such as e-commerce or academic 
or tourist ones. 


QEM (Quality Evaluation Method) [8] was proposed 
through a case study on museums, then it was applied 
to academic domain, specifying about eighty directly 
measurable attributes and four main areas ( Usability, 
Functionality, Site Reliability and Efficiency). This model 
presents a very specific domain-dependent approach. 


The 2QCV3Q-model [5] adds the seventh locus 
Quibus Auxiliis (Feasibility) to the 2QCV2Q-model [6] 
focused on six main dimensions corresponding to six 
loci: Quis (Identity), Quid (Content), Cur (Services), 
Ubi (Individuation), Quando (Management), Quomodo 
(Usability). This model was introduced to marketing 
purposes and was mainly applied in tourist 
environment. 


MiLE (Milano-Lugano Evaluation method) [1] tries to 
combine inspection method with empirical testing. 
MiLE+ [11] is the evolution of MiLE method that 
introduces a clear distinction between the application- 
independent and the application-dependent analysis. 
Moreover it proposes a specific activity, called Technical 
Inspection, which aims at analysing the application- 
independent aspects and two evaluation activities for 
the application-dependent ones, called User experience 
inspection and Scenario-based user testing. This model 
is excellent about some aspects but doesn’t consider 
accessibility for people with disabilities at all. 


MINERVA (MInisterial NEtwoRk for  Valorizing 
Activities in digitization) project [7] represents an 
important initiative to encourage the application of ten 
quality principles (Transparent, Effective, Maintained, 
Accessible, User-centred, Responsive, Multi-lingual, 
Interoperable, Managed, Preserved), includes assistive 
technologies features, but it’s aimed at cultural 
websites (museums, libraries, archives and other 
cultural institutions). 


Polillo [9] presented an evolution of ETNOTEAM 
model [2], including seven dimensions (Architecture, 
Communication, Functionality, Content, Management, 
Accessibility, Usability). 


Signore presented a rigorous approach [10], based 
on five measurable characteristics (Correctness, 
Presentation, Content, Navigation and Interaction). He 
started his method from the limitations of other 
approaches, but he developed a model, aimed at the 
realization of an automated software, that is focused 
mainly on the technical aspects that can be measured, 
giving less importance to perceivable and 
communicative features. 


In this context, we worked out a new quality model 
that aims at being an useful tool to realize a quick 
evaluation that takes into account three different 
quality levels and therefore different execution times, a 
very important aspect for project managers. Thus our 
model represents a complete method of websites 
quality in use evaluation that allows a constructive 
collaboration among web designers, developers and 
managers, and a growing consideration about quality in 
the web projects. 


Our project 

After the research work about the existing quality 
models, we summarized all the information collected 
and extrapolated a model from the best characteristics 
of each considered methodology. Rarely different 
models use the same term to refer to semantically 
equivalent characteristics. Often there are 
characteristics, called with different names, that have 
similar meaning or recall the same concept or belong to 
the same context. So, for this reason, in the first step, 
we assembled all attributes of all models and, in the 
second step, we separated them in a limited number of 
groups on the basis of their semantic meanings. At this 
point, we eliminated the existing repetitions and we 
well-finished and optimized the groups. The output 
consists of six dimensions, represented by the six 
groups assembled, listed below. 


Interface Communication (IC) 


1. Using clear, transparent, specific and univocal 
labels or titles 


2. Using intuitive icons and symbols that follow 
standards and familiar conventions to web users, or 
whose meaning is easy to understand anyway 


3. Using sharp images, of good quality and pertinent 
to site’s communication purposes 


4. Page layout: giving importance to elements 
distribution, taking into account possible 
conventions about their position; avoiding 
information overload or overcrowded pages; 
verifying page scannability and grouping adequacy 
of information units 

5. Preserving coherence and stability of main 
elements within the pages 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


Visual identity: importance of Brand or personal 
identity; explicating site’s property and mission, 
also using graphic style (colour, image) 

Using colours to identify and distinguish website’s 
areas and to attract user’s attention on page’s 
different element 


Choosing adequate colours for the combination 
text/background not to make reading difficult 


Choosing font type and size to facilitate reading 


Identifying links with clear, conventional and 
coherent symbology to easily distinguish them 
among other elements 


Showing link state change using a colour change 


Using icons and widgets consistent and pertinent 
with site’s colours and other graphical elements 


Text layout: verifying paragraph division and text 
structure and alignment 


Using graphical style and colours in coherent way 
within all pages 

Multimedia elements (video, flash animations, etc): 
verifying their consistence, adequacy and 
coherence with site’s graphical behaviour 


Interface predictability: using interactive elements 
(symbols, icons, links, images, buttons, etc) able to 
anticipate their content and the interaction effects 


Realizing a simple interface to allow the user to 
learn site’s communication strategy quickly and to 
remember site’s structure when he or she returns 
on the website after a period of not using it. 


Content (CO) 


1. 


2. 


Information quality: verifying if content is accurate, 
reliable, complete, detailed, impartial, correct 


Verifying content currency 


Considering website’s domain coverage, depending 
on site’s owner and user’s aims 


Verifying content conciseness 

Verifying content selection and relevance 

Making sure of content comprehensibility and 
readability (choice of language, terms and text 


style pertinent to users’ profile; use of bullet or 
numbered lists, etc) 


Providing at least a basic service in another 
language 


Choosing multimedia content (image, video, audio, 
photo, etc) so that is strictly connected with the 
text to provide further adequate information. 


Navigation (NA) 


1. 


Site's structure (architecture and topology): 
verifying its coherence with content and if it makes 
navigation easy 


Helping the user not to lose his/her orientation in 
the website (using breadcrumbs, link titles and 
other clues to communicate always where the user 
is and where he or she will be led) 


Using adequate, easy and intuitive navigation tools 
(bars, search engine, site map, indexes, guided 
tour, image map, etc) 
Supporting backward navigation (“Go back” button, 
history, back to Home) 


Allowing the user to access each site’s page in few 
click. 


Management and Accessibility (MA) 


1. 


Hardware and software requirements: verifying 
browsers and platforms (WebTV, mobile phone, 
PDA) compatibility; avoiding use of plug-ins and 
proprietary extensions 


2. Controlling site’s presence on the most famous 
search engines 


3. Verifying code correctness (alt attributes, 
equivalent text for multimedia elements, HTML and 
CSS validation) 


4. Optimizing pages view time (download + parsing + 
+ rendering) and media streaming 


5. Maintenance: site’s monitoring, assuring 
uninterrupted availability and technical accessibility 


Assuring functionality’s adequacy and correctness 


Using security policy and guaranteeing personal 
information correct use 


8. Errors management: verifying system reactions, 
error scripts comprehensibility, deleting or 
repairing broken links and under construction 
pages. 

Interactivity (IN) 

1. Assuring user interaction functionality adequacy: 
naturalness, effectiveness, precision, transparency 
about user's actions consequences, recovery (undo 
availability) 


2. Considering users’ involvement and contribution 
(comments, hints, guestbook, community, etc) 

3. Providing help and contact information 
Assuring quick, pertinent, polite answers to users’ 
questions and doubts 


5. User satisfaction: making the website well- 
accepted, pleasant and easy to use; choosing the 
best position and width for elements and links 
(using, for example, Fitts’ law). 

Accessibility for people with disabilities (AD) 

1. Conformity to W3C guidelines 


Web Q-Model 

Thinking about Von Dran et al. [12] and Kano Model 
[4] for the customer expectations of service or product 
quality, and after the formulation of our model's 
attributes, we also decided to classify them in three 
levels: Basic, Normal and Exciting. For each level we 
associated a symbol, in particular Q for the Basic, QQ 
for the Normal and QQQ for the Exciting. This choice is 
made to differentiate the attributes on the basis of their 
importance and essentiality in a good quality website 
design. 


We present a general model, even if each feature 
can be more or less important depending on the 
purpose of the site or the users’ profile. Moreover our 
model is characterized by a clearly visible scalability, 
because the different attributes will update their level 
membership depending on time, technology evolution 
and social customs. 


Web Q-Model Classification 


Qe) QQ QQQ) 





IC 1,2,4,6,7,8,9,10,17 |3,5,11,12,13,14 |15,16 





co {1,6 4 





NA [1,2 3,4 5 





MA |1,3,6,7,8 





IN 1,3 4 2,5 























AD We consider W3C guidelines and checkpoints and 
their division in three level of priority. [13] 














(*) Q=Basic level, the minimum acceptable, that is 
those features that the user takes for grant. 
QQ=Normal level, consciously stated user needs, 
whose absence will cause disappointment or a sense of 
disadvantage. QQQ=Exciting level that includes those 
characteristics that delight users because they don’t 
know their existence or don’t feel a conscious need for 
them. 


Conclusions and future works 

In this work, we proposed a new quality evaluation 
model for websites that is intuitive, scalable and easy 
to apply, facilitating developers and designers’ work 
and the dialogue among them and the managers. In 
particular, the classification in three Q-/evels will enable 
managers to propose different quality approaches to 
their customers, corresponding to different prices, work 
times and resources to employ. For the future, we want 
to submit a questionnaire to a large number of typical 
users to render our classification more shared and to 
improve our model. 
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